Digital cameras and other imaging devices are becoming widely used in both consumer and industrial applications. Such devices acquire images by way of an imaging array. An imaging array typically comprises an array of radiation-sensing elements or “pixels”. The pixels are typically arranged in rows and columns. Arrays of Charge-Coupled Devices (CCDs) and Active Pixel Sensor (“APS”) arrays are two examples of imaging arrays.
The resolution of images acquired by an imaging array depends upon the density of pixels in a sensing area of the imaging array. Providing more pixels within the same sensing area provides higher-resolution images. An imaging array in a digital camera may have several million pixels, for example.
The probable number of defective pixels in an imaging array tends to increase with the number of pixels in the imaging array. Pixel failure may occur when the imaging array is fabricated, later when the imaging array is exposed to a stressful environment or as a result of components degrading over time. Like all microelectronics, as imaging arrays age their components can start to fail. General electronic devices tend to produce failures either during their first six months (called infant mortality) or after a few years (typically three years or more). In many devices (computers, cell phones) these failures render the device unusable. However, in camera systems, failures of some pixels are not necessarily catastrophic because such failures can often be corrected for.
In an imaging array having a large number of pixels, it is likely that at least a few of the pixels will be defective. For this reason, many digital cameras employ software correction. The software correction may replace the outputs from pixels determined at fabrication time to be defective with corrected outputs. The corrected outputs are often weighted averages of outputs from adjacent non-defective pixels.
In the fields of amateur and professional photography, pixel failures have become more noticeable as digital camera use has become more widespread. Such failures tend to appear as defective pixels or areas of the picture which are not correct. These can be quite noticeable and will tend to accumulate over time. Higher-end cameras, such as digital single lens reflex (“SLR”) cameras, are quite expensive. It is undesirable to have to replace such equipment because a few pixels have become defective. However, unless the defective pixels are identified and corrected for the presence of the defective pixels degrades images obtained with the camera.
Defective pixels in imaging arrays can cause particularly acute problems in imagers operating at wavelengths outside of the visible spectrum. For example, fabrication processes for infrared imagers are often more complex than those for visible light imagers. Thus infrared imagers often have defect rates significantly higher than visible imagers. Similarly, digital medical X-ray imagers require large areas and are fabricated using processes that can make them more prone to faults. Further, obtaining accurate images can be important in the medical and scientific areas in which such imagers are applied.
Faults in imaging systems are becoming more prevalent as the areas and pixel counts of imaging arrays increase while pixel size shrinks. Consequently, identifying and correcting for pixel faults is crucial to improve the yield (during fabrication) and reliability (during operation) of imaging arrays.
To correct defective pixels with either software or hardware techniques, the location of defective pixels must be determined. The simplest defect models assume a pixel can be “good” (non-defective), “stuck high” (always bright or white), or “stuck low” (always dark). More advanced fault models include low- and high-sensitivity pixels and hot pixels.
It is known to detect defective pixels in an imaging array at fabrication time by taking two exposures with the array. A dark-field (no illumination) exposure detects pixels that are stuck high. A light-field (illumination level near saturation) exposure detects pixels that are stuck low or have low sensitivity. Dark-frame exposures can typically be obtained in the field. Uniform light-field illumination (or any uniform illumination) is difficult to obtain without additional hardware.
It is more difficult to detect pixels which respond to exposure but have defects which cause the pixels to have sensitivities that are significantly different from the nominal pixel sensitivity. More advanced techniques can detect faults such as high-sensitivity pixels. A high-sensitivity pixel provides a low output value when it is not illuminated but rapidly saturates at modest illumination levels. High-sensitivity pixels are thus not detectable with two simple illumination fields.
Most cameras employ colour imaging arrays. Colour imaging arrays present further complications. A colour imaging array typically comprises red-, green- and blue-filtered pixels. The pixels may be arranged in a pattern called the “Bayer Matrix pattern” as disclosed in Bayer, U.S. Pat. No. 3,971,065. Defects will typically be distributed among pixels at each colour site on the matrix, and hence defect detection must take into account the colour filter pattern of the imaging array.
Frame et al., U.S. Pat. No. 4,590,520 discloses detecting pixel faults by performing a post-production calibration to obtain “correction coefficients” for each pixel. A pixel is identified as being bad if its correction coefficient is not close to the values of the correction coefficients of neighbouring pixels.
Kagle et al., U.S. Pat. No. 6,819,358 discloses identifying defective pixels by means of light-, dark- and intermediate-range exposures. This technique can identify some pixels that have abnormal sensitivities (high or low).
St. Clair, U.S. Pat. No. 4,193,093 discloses that stuck-high pixels can be identified in a video imager by comparing the pixel values to the saturated video signal. However, in any complex image, this would falsely detect as being stuck-high many pixels that are in saturation due to the exposure of the image.
Peairs et al., U.S. Pat. No. 5,694,228 discloses methods for identifying defects in an imager based upon digital images of scanned documents. The methods involve incrementing a counter for potential defects when a pixel location has the same colour in more than a threshold number of the digital images. Otherwise the counter is reset or decremented.
Jin et al. Modeling and Analysis of Soft-Test/Repair for CCD-Based Digital X-ray Systems IEEE Trans. Instrumentation and Measurement vol. 52, pp. 1712-1721, December 2003 compare the difference between a pixel value and the average of pixel values for eight nearest-neighbour pixels to high or low threshold values to identify faulty pixels.
Tan et al., U.S. Pat. No. 6,381,357 discloses identifying pixel faults using the difference between a pixel value and the average of pixel values for the four or eight nearest-neighbour pixels combined with sequential probability ratio testing on multiple images to identify defects.
These methods can be susceptible to incorrectly identifying good pixels as being defective (i.e. creating significant numbers of false positives). In some reported cases these methods can give as many, or more, false positives as correctly-identified faults. In some cases these methods have been found to yield two or more times as many false positives as the actual number of defective pixels.
False positives can be as detrimental to image quality as defective pixels. If the outputs of pixels falsely identified as faulty are replaced by interpolating the output from neighbouring pixels, as understood by practitioners of the art, the image quality can be reduced. This is particularly true if the pixel lies in an area where the image changes rapidly, such as at the edge of an object.
There remains a need for robust methods and apparatus for detecting defective pixels in imaging arrays. There is a particular need for such methods and apparatus that can be performed on-line (i.e. while a device is operating) to detect faults that occur after fabrication.